In large language models (LLMs), certain neurons can store distinct pieces of knowledge learned during pretraining. While knowledge typically appears as a combination of relations and entities, it remains unclear whether some neurons focus on a relation itself -- independent of any entity. We hypothesize such neurons detect a relation in the input text and guide generation involving such a relation. To investigate this, we study the Llama-2 family on a chosen set of relations with a statistics-based method. Our experiments demonstrate the existence of relation-specific neurons. We measure the effect of selectively deactivating candidate neurons specific to relation $r$ on the LLM's ability to handle (1) facts whose relation is $r$ and (2) facts whose relation is a different relation $r' \neq r$. With respect to their capacity for encoding relation information, we give evidence for the following three properties of relation-specific neurons. $\textbf{(i) Neuron cumulativity.}$ The neurons for $r$ present a cumulative effect so that deactivating a larger portion of them results in the degradation of more facts in $r$. $\textbf{(ii) Neuron versatility.}$ Neurons can be shared across multiple closely related as well as less related relations. Some relation neurons transfer across languages. $\textbf{(iii) Neuron interference.}$ Deactivating neurons specific to one relation can improve LLM generation performance for facts of other relations. We will make our code publicly available at https://github.com/cisnlp/relation-specific-neurons.

The investigation into Sunday's death of "Star Trek" actor Anton Yelchin, 27, in a 2015 Jeep Grand Cherokee rolling accident continues. But signs point to a problem that Fiat Chrysler Automobiles has had for years with its electronic automatic gear shifters -- namely that some of its cars have shifters that can leave drivers believing they've been put safely into park. Instead, the cars can roll away, causing accidents, injuries and even death. And while the problem vexing more than 800,000 recent Jeep Grand Cherokee SUVs, Dodge Charger muscle cars and Chrysler 300 luxury sedans isn't widespread across the industry, the increasing use of electronics for crucial vehicle functions is posing new challenges for automakers, and new risks for drivers. "There's this rush to electrify every aspect of a vehicle," said Byron Bloch, an auto safety expert with 40 years of experience inspecting vehicles and testifying on accidents.